H2HH: Introduction to the Hoffman2 Cluster¶

Raffaella D'Auria, PhD¶

Learning Outcomes¶

  • what is the Hoffman2 Cluster
  • what can the Hoffman2 Cluster do for you
  • navigating the documentation / policies
  • getting an account on Hoffman2
  • connecting to the Hoffman2 Cluster and opening a graphical application
  • unix command line 101
  • where to look for applications already available on Hoffman2

Screen%20Shot%202022-02-07%20at%2012.45.17%20PM.png

Nomenclature¶

Compute node:¶

  • A compute node is a server (computer) with multiple computing cores that can execute as many simultaneous processes or threads as many cores it has

Computing core:¶

  • A basic unit of computation within a CPU

Login node:¶

  • A server designed to ensure connectivity between the "world" and the computing nodes on the cluster

Nomenclature (cont'd)¶

Computing Cluster:¶

  • A computing cluster is made of individual nodes (computers) which are interconnected between themselves and which aggregate power can be harnessed to address problems which nature requires either distributed or capacity computing

What can the Hoffman2 Cluster do for you¶

High Performance Computing¶

  • distributed computing (multiple computing cores across multiple compute nodes)
  • shared-memory computing (multiple cores on the same compute node)
  • data-intensive computing
  • high-memory computing

Capacity Computing¶

  • multiple computing cores available to carry out very many simulations
  • execute concurrent (synchronous) or indeopendent (asynchronous) simulations

What can the Hoffman2 Cluster do for you (cont'd)¶

Licensed software availability¶

  • Abaqus
  • Intel Parallel Cluster Studio
  • Maple
  • Mathematica
  • MATLAB

Licensed software management¶

  • bring your own licensed software (COMSOL, Ansys, FDTD, etc.)

User Support/Project Collaboration/Training¶

Navigating the documentation¶

https://www.hoffman2.idre.ucla.edu/

Screen%20Shot%202022-02-07%20at%204.35.52%20PM.png

The Hoffman2 Cluster - About¶

https://www.hoffman2.idre.ucla.edu/About/System-overview.html

Screenshot%202023-10-10%20at%209.22.43%E2%80%AFPM.png

Getting an account on Hoffman2¶

https://www.hoffman2.idre.ucla.edu/Accounts/Requesting-an-account.html

Screenshot%202023-10-10%20at%208.33.07%E2%80%AFPM.png

Requesting a user account¶

https://www.hoffman2.idre.ucla.edu/Accounts/Requesting-an-account.html#applying-for-a-user-account

Screenshot%202023-10-10%20at%208.42.29%E2%80%AFPM.png

Requesting a sponsor account¶

https://www.hoffman2.idre.ucla.edu/Accounts/Requesting-an-account.html#register-as-a-sponsor

Screenshot%202023-10-10%20at%208.41.34%E2%80%AFPM.png

Sponsor responsibilities¶

https://www.hoffman2.idre.ucla.edu/Accounts/Sponsors-managing-your-group.html#sponsor-responsibilities

Screenshot%202023-10-10%20at%208.57.01%E2%80%AFPM.png

Managing your account¶

https://www.hoffman2.idre.ucla.edu/Accounts/Users-managing-your-account.html#password-processes

Screenshot%202023-10-10%20at%209.10.20%E2%80%AFPM.png

Password Processes¶

https://www.hoffman2.idre.ucla.edu/Accounts/Users-managing-your-account.html#password-processes

Screenshot%202023-10-10%20at%209.12.57%E2%80%AFPM.png

For sponsor: managing users in your group¶

https://www.hoffman2.idre.ucla.edu/Accounts/Sponsors-managing-your-group.html#managing-your-users

Screenshot%202023-10-10%20at%209.14.37%E2%80%AFPM.png

Connecting to the Hoffman2 Cluster¶

https://www.hoffman2.idre.ucla.edu/Using-H2/Connecting/Connecting.html

Screen%20Shot%202022-02-07%20at%204.55.02%20PM.png

Connecting via a terminal and an SSH client¶

https://www.hoffman2.idre.ucla.edu/Using-H2/Connecting/Connecting.html#connecting-via-terminal-and-ssh

Screen%20Shot%202022-02-07%20at%208.30.58%20PM.png

Terminal and SSH client (Linux and MacOS)¶

Screen%20Shot%202022-02-07%20at%207.11.00%20PM.png

Terminal and SSH client (Linux and MacOS) (cont'd)¶

Screen%20Shot%202022-02-07%20at%207.10.41%20PM.png

Terminal and SSH client (Linux and MacOS) (cont'd)¶

Screen%20Shot%202022-02-07%20at%207.11.32%20PM.png

Connecting via a terminal and SSH for the first time¶

Screen%20Shot%202022-02-07%20at%209.01.23%20PM.png

Connecting for the first time¶

https://www.hoffman2.idre.ucla.edu/About/FAQ/FAQ.html#connecting-for-the-first-time

Screenshot%202023-10-10%20at%209.27.29%E2%80%AFPM.png

Hoffman2 Cluster public hostkey fingerprints¶

https://www.hoffman2.idre.ucla.edu/About/FAQ/FAQ.html#table-h2-public-host-classes

Screenshot%202023-10-10%20at%209.29.35%E2%80%AFPM.png

Connecting from Windows with terminal and SSH¶

https://www.hoffman2.idre.ucla.edu/Using-H2/Connecting/Connecting.html#connecting-via-terminal-and-ssh

On Windows one of the following applications should be installed:

  • MobaXterm terminal for Windows with X11 server, SSH client & network tools
  • GitBash terminal with SSH client
  • Windows Subsystem A Windows native compatibility layer for running Linux binaries on Windows (available from Windows 10)
  • PuTTY terminal with SSH client
  • Cygwin a large collection of GNU and Open Source tools which provide functionality similar to a Linux distribution on Windows

X Window Sytems/X11¶

The software that allow displaying of graphical applications on a unix-like system and that is designed to work on a server-client mode (you can connect to it and forward graphical applications to your local computer)

  • For example: forward the MATLAB desktop GUI from Hoffman2 to your local computer

Screen%20Shot%202022-02-07%20at%2011.23.57%20PM.png

Opening graphical applications on Linux (Hands-on)¶

  • X11 server is natively available
  • forwarding of graphcail applications for remote display is enabled via:
    ssh -X joebruin@hoffman2.idre.ucla.edu
    substitute joebruin with your Hoffman2 username

Opening graphical applications on MacOS (Hands-on)¶

first time only:¶

  • install Xquartz (the X11 server for MacOS)
  • open the terminal app and before connecting to Hoffman2 issue:
defaults write org.macosforge.xquartz.X11 enable_iglx -bool true
  • reboot your local computer

connect with:¶

  • forwarding of graphcail applications for remote display is enabled via:
    ssh -Y joebruin@hoffman2.idre.ucla.edu
    substitute joebruin with your Hoffman2 username

Opening graphical applications on Windows¶

no need to install other software then any of the following:¶

  • MobaXterm
  • Windows Subsystem for Linux
  • Cygwin

need to install an X11 window server if using either of:¶

  • GitBash
  • PuTTY

a possible X11 Window server:¶

  • Xming

Opening graphical applications on Windows using MobaXterm (Hands-on)¶

Screenshot%202022-02-08%20074952.png

Opening graphical applications on Windows using MobaXterm (cont'd)¶

Screenshot%202022-02-08%20075046.png

Opening graphical applications on Windows using MobaXterm (cont'd)¶

Screenshot%202022-02-08%20075130.png

Opening graphical applications on Windows using MobaXterm (cont'd)¶

login.png

Opening graphical applications on Windows using MobaXterm¶

xclock.png

Unix command line 101¶

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html

Screen%20Shot%202022-02-08%20at%208.49.56%20PM.png

Unix command line 101: Navigation (Hands-on)¶

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html#navigation

  • Try these commands:
ls
ls -a
ls -l
ls -lat
ls -latr
pwd
cd $SCRATCH
pwd
timestamp=`date "+%F_%H-%M"`; mkdir $HOME/H2HH_$timestamp; cd $HOME/H2HH_$timestamp
pwd
rmdir $HOME/H2HH_$timestamp
ls $HOME/H2HH_$timestamp

Move to Hands-on notebook¶

Unix command line 101: Environmental Variables (Hands-on)¶

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html#unix-environment-environmental-variables

  • Some important environmental variables:
$PATH
$LD_LIBRARY_PATH
$HOME
$SCRATCH  # Hoffman2-specific
$SHELL
  • to check the content of an environmental variable issue:
echo $HOME

Move to Hands-on notebook¶

Unix command line 101: Working with files (Hands-on)¶

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html#working-with-files

  • Try these commands:

    cat $HOME/.bashrc
    less $HOME/.bashrc   # type q to interrupt
    more $HOME/.bashrc   # type q to interrupt
  • Check the first few lines of a file:

    head $HOME/.bashrc
    head -n 2 $HOME/.bashrc
  • Check the last few lines of a file:

    tail $HOME/.bashrc
    tail -n 2 $HOME/.bashrc
  • Check the last lines of a file as it is being written:

    tail -f $SGE_ROOT/$SGE_CELL/common/accounting   # Control-C to interrupt

Move to Hands-on notebook¶

Unix command line 101: Editing files (Hands-on)¶

https://www.hoffman2.idre.ucla.edu/Using-H2/Command-line/Unix-command-line.html#editing-files

  • Non graphical editors:

    nano 
    emacs
    vi
  • Graphical editors:

    gedit &
    emacs &
    gvim &

Getting work done on the cluster¶

Screen%20Shot%202022-02-08%20at%209.39.32%20PM.png

On which resources will your job run?¶

https://www.hoffman2.idre.ucla.edu/Using-H2/Computing/Computing.html#computational-resources-on-the-hoffman2-cluster

Highp vs shared vs campus resources:¶

https://www.hoffman2.idre.ucla.edu/Using-H2/Computing/Computing.html#highp-vs-shared-vs-campus-jobs

  • highp refers to the use of group owned compute nodes
    • users can run jobs for up to 14 days
    • only for users in group who own resources
  • shared refers to the use of temporarely unused group owned compute nodes
    • users can run jobs for up to 24 hours
    • only for users in group who own resources
  • campus refers to compute nodes owned by IDRE and made available to the UCLA community
    • users can run jobs for up to 24 hours

On which resources will your job run? (cont'd)¶

https://www.hoffman2.idre.ucla.edu/Using-H2/Computing/Computing.html#jobs-and-resources

Working interactively on the cluster¶

Where to look for applications already available on Hoffman2¶

Running non interacive workflow (batch jobs)¶